The present invention relates generally to the field of image capture and, more particularly, to panoramic video cameras, camera systems, and methods that facilitate handling multiple video streams while tracking an object.
Panoramic video cameras are known to capture a field of view (FOV) of 360° about an optical axis of the lens or lenses used in the cameras. Such FOV is typically referred to as the “horizontal FOV” of the camera. Panoramic video cameras can also simultaneously capture an FOV about an axis orthogonal to the optical axis of the camera lens(es). This additional FOV is typically referred to as the “vertical FOV” of the camera. The vertical FOV may exceed 180° when the camera includes one or more ultra-wide angle lenses. The combination of the horizontal FOV and the vertical FOV provides the overall FOV of the panoramic video camera (e.g., 360°×180°, 360°×270°, and so forth). A wide overall FOV permits the camera to capture environmental information about a physical region surrounding the camera. Accordingly, a single panoramic video camera placed in the center of a meeting table is capable of capturing imagery for all participants sitting around the table.
One common use case for a panoramic video camera is to mount the camera on a moving object, such as a person, bicycle, or automobile, to capture imagery of an activity, such as skiing, surfing, bike riding, auto racing, and the like. A goal of such use is to permit playback of the captured video on a display that enables a viewer to become immersed in the experience. A user often is provided controls to alter a view of video playback, where the playback includes a relative center. That is, playback typically provides a horizontal FOV viewing segment of approximately 110° of an available 360° arc. For playback, a center for this viewing segment must be defined, which by default is typically a vector consistent with a relative motion of the camera itself. So, when the camera moves in a “northeast” direction, a viewing segment center is set to “northeast” by default. Thus, camera motion functions as a “true north” for the camera for playback purposes, which is often determined by an internal motion sensor, such as a gyroscope, compass, or other such sensor.
Often, there is a desire for real-time or near real time playback (e.g., live streaming) of video from a panoramic video camera. This rapid response complicates intra-camera processing tremendously because it requires camera processor operations to be split between image capture and playback. To ensure effective end-user experiences at playback, some minimal overhead is necessary, which is often handled internally by camera hardware. For example, the horizontal FOV viewing segment for playback video is a sub-portion (e.g., 110° degree arc) of the horizontal FOV for the captured image content (e.g., 360° arc). Additionally, the horizontal FOV viewing segment boundaries may require adjusting to stabilize playback. That is, because the camera itself is in motion due to being mounted to a dynamically moving object, a smoothing function may need to be applied to compensate for camera-based motion. Overall, processor intensive digital signal processing (DSP) operations must be minimized to minimize latency between video capture and playback. The need to minimize processing has resulted in a conventional opinion that for dynamically moving cameras, the only viable direction able to be set as “true north” for playback purposes results from hardware sensor data, such as sensor data indicative of camera motion. Conventionally, playback direction either matches camera movement direction or is fixed. For example, a “center” for the playback view may be fixed to a compass direction regardless of camera motion. Alternatively, a movement direction of the camera itself may be used as a “center” to define viewable playback boundaries. While restricting playback direction to internal sensor information is sufficient in many instances, instances exist where a greater flexibility of playback direction is desired, yet not conventionally possible.
Those skilled in the field of the present disclosure will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein. The details of well-known elements, structure, or processes that would be necessary to practice the embodiments, and that would be well known to those of skill in the art, are not necessarily shown and should be assumed to be present unless otherwise indicated.